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ABSTRACT 


The first section of this paper is concerned with determining the 
required lower confidence limit that must be met by testing after a 
missile system becomes operational. Some of the costs of making 
decisions about the required system reliability lower confidence limit 
are discussed. Two possible cost effective models for determining 
the optimum test size are suggested. 

The second section of this paper is concerned with the effects of 
changing the number of missiles tested in each year while maintaining 
the total number of systems tests constant over the missile system's 
projected operating life. In other words, in this section; the effects 
of testing heavily in the first years versus testing heavily at the end 
of the system's life or versus testing uniformiy throughout the life of 
the system are compared. For this comparison, the sum of the vari- 
ances of the estimators determined from the results of the tests con- 
ducted in each year 1s obtained. This value is compared for six 
representative distributions of testing throughout an estimated system 


life of ten years. 
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ie “INTRODUCTION 


There are three types of tests for our current Polaris missile 
System: 

1. The Demonstration and Shakedown Operation - DASO; 

Z. Operational Tests - OT; 

3. Follow-on Operational Test - FOT. 

DASO represents the first test of the weapon system in its totality. 
For this test, the missiles are carefully prepared and then fired under 
strigent conditions. The results of DASO firings are not included in the 
data for determining the weapon system reliability. 

The OT program is a one-time test program with the specific ob- 
jective of determining weapon system reliability. It is conducted under 
the most realistic conditions possible, The system is tested in sucha 
way that the entire operational system, including communications, 
authentication, navigation, and accuracy, is tested; and the observed 
Success rate 16 computed. 

The FOT is conducted in the same manner as the OT with the 
specific objective of insuring that the weapon system reliability factor 
determined in the OT program is, in fact, still valid in the years 


following the Operational Tests. 


This paper is concerned with determining the optimal OT and FOT 
test sample sizes. Much of the work presented here was started this 
Summer at the Office of Programs Appraisal under the direction of 
Captain D. A. Paolucci. In addition, some of the ideas introduced 


come from unpublished notes of Captain Paolucci. 


2. SUMMARY AND CONCLUSIONS 
2.1 A Model for Maximizing the Number of Reliable Missiles in 

Inventory 

In the final analysis, the number of missiles from a system counted 
on by the strategic planner will be determined by the number of missiles 
available and the 90 percent lower confidence pas for system 
reliability. In other words, if the 90 percent lower confidence limit of 
reliability is .7 and there are N missiles available, the strategic 
planner will count on . 7N when targeting the system. 

Assuming that the observed success rate is relatively constant, 
the lower confidence limit of reliability will increase every time a 
missile is successfully tested. By observing Figure 2.1.1, it can be 
seen that the marginal réturn for each test reaches a near zero point 
when the lower confidence limit comes within , 11 to . 09 of the observed 
success rate. [his observation might lead to the criterion that testing 
should continue until the lower confidence limit of reliability is within 
sll -to..09 of the observed success rate, 

it can be shown, See section 3.1, that Wwsing this eriterion for 
determining test sizes can, under certain circumstances, result in 4 

To avoid messy notation and confusing the problem, this 

paper will always use the 90 percent confidence level, This choice of 


90 percent is completely arbitrary and may not necessarily reflect 
Navy policy. 
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smaller number of reliable missiles counted on by the targeter than 
would be counted on if fewer missiles were tested. 

For this reason, the following alternate model for determining the 
required test size to estimate missile system reliability is suggested. 
Choose n to maximize Q in the following expression. 

O-= LIM =n), where: 


L, = the 90 percent lower confidence limit determined 
by n and the observed success rate; 


M = the total number of missiles purchased; 
nh = the number of trials, 

Using this criterion to determine the test size results in testing 
about 33 missiles if the observed success rate is in the area of .7 to 
.9. The criterion mentioned earlier requires approximately 35, 46, 
and 49 missiles for observed success rates of .9, .8, .7, respectively. 

A cost effective model for determining the test size to most 
effectively increase the reliable yield of the expected number of missiles 
deployed is presented in section 3,2, This model uses the marginal 
cost Of increasing the expected yield of the Polaris missile system by 
procurement as the cut-off point for spending on missile testing. 

The cost information necessary to obtain an explicit number for 
test sizé using this model is not known to the author, Therefore, no 


See section 3.1 for mathematical justification of this 
statement, 


further discussion of this model will be presented in this section except 
to note that the figures necessary to obtain an explicit solution are 


obtainable. 
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2.2 Brief of Method Used in Investigating Models One, Two, and Three 

The present doctrine used to determine FOT test sizes is to use 
the same size test that is necessary for the OT. Therefore, for the 
next ten years somewhere between 300 and 400 missiles would be 
programmed for expenditure during OT's and FOT's. 

To investigate some of the effects on reliability estimation by 
changing the yearly FOT test size, six different sample plans that 
distribute the testing throughout the years in different ways were con- 
sidered, These sample plans are given in Table 2, c, 1, 

To avoid confusion in this section, all testing done in year i will 
be designated as test 1; and the individual shots will be designated as 
trials. Each different way of distributing the tests among the years 
will be called a sample plan. 

To compare the different sample plans, the sum of the variances 
computed in each year will be obtained for each sample plan. In other 
words, the variance of the estimator of reliability will be obtained 
under a sample plan for year one. Then, using the same sample plan, 
the variance of the reliability estimator will be obtained for years 
two through ten. Once these values have been obtained, the ten of them 
will be added together and the sum will be called the sum of the 
variances. 


The reason for the criterion of comparing the sum of the variances 
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can be likened to the quest for a ''minimum variance estimator" in 
many conventional estimation problems. 

The sum of the variances depends on the different test sizes, the 
initial reliability (Pp), and the total number of missiles tested in the 
ten years. An attempt has been made to investigate the different sample 
plans over a plausible range of values for Po and N, the number of 
muassiles scheduled for testing during the first ten years of operational 
life. To this end, the sum of the variances for each sample plan is 
computed using all possible combinations of .7, .8, .9 for Po: and 200, 
300, and 400 for N. 

Perhaps a little more explanation 1s:merited here., Assume that the 
sum of the variances of sample plan one is being computed. Now look 
at the situation where ee (and N=<00, When N= 200, sample 
plan one requires that 20 trials of the system be conducted in the first 
year. From this, an expression for the variance of the estimator can 
be obtained. Likewise, an expression for the variance of the estimator 
in years two through ten can be obtained. Once the variances for all 
the years have been obtained, they are added together and the number 
is recorded as the sum of the variances under sample plan one, where 
Ba 7 and N = 200. Next, sample plan two will be used; and the 
sum of the variances under this sample plan, when Py =- T-and N =-200; 


will be obtained. 
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We will continue in this fashion until the sum of the variances 
when Py =- 7 and N = 200 have been determined for sample plans one 
through six. Then; the next combination of Po and N, say Pa ce 
N = 300, will be considered. In the same manner as before, the sum 
of the variances for all the sample plans will be obtained, The compu- 
tations will continue until the sum of the variances for each sample 
plan has been obtained for all possible combinations of the three values 
of Po and N. Having obtained these variances, those sampling plans 
with the associated smaller variance values would be superior to those 
with larger variances from the point of view of having accurate esti- 


mates of missile reliability. 
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2.3 Determining FOT Sample Size Using Model One 
Consider a collection of N missiles that are characterized by life- 


times X_, X 


; arte BO where X, represents the time from initial 
i 


“ inspection and release until the i item deteriorates to an unacceptable 
state. If it is assumed that the initial states of the missiles are the 
Same, then the X's are non-negative independent, identically distributed 
random variables, Suppose that there are ten distinct times that ob- 
servations are to be taken from the collection of missiles. Also, assume 
that K. items are to be observed at time t.. These observations can be 


summarized by the random variables Y,, Y re ts where: 


| ena oe 
i 


y - { 1 if the missile system fails 


O if the missile system is successful 
On the basis of this information, it is desired to estimate pay 1 
P (a missile's lifetime is greater than ae ior each t. = j, 
j = 1, 2,24, 710. The estimation procedure which 1s proposed has 


the following jorm, For each j, the estimate of ae! is given by 


1s 


where: 


K. nN, L, J eee 


Moe = Oe laa - : 
1 


0 1 : 


1 
ih 


K, = noof trials in.vear x 
lif test 1 tails 
0 if trial iis successful 

This estimator provides continuity to the estimates of reliability 
over the ten years, see section 3.3. However, this estimate is 
generally more optimistic in the early years and more pessimistic in 
the out years than the Maximum Likelihood Estimator. The MLE 
mentioned here is the MLE of reliability using only the information 
obtained from the current test. Note that Model One does take past 
data into account. 

To investigate Model One, it was assumed that the probability of 
a successiul' trial at any piven time t could be expressed by the expo - 
nential function: 

P(t) = Poe nee where b is an unknown parameter, 

The sum of the variances was then computed for all combinations of 
Po and N, as explained before, In addition, these values were computed 
for values of b ranging from .002to.2. The results of these compu- 
tations are displayed in the graphs on pages 20 through 37. 

On these graphs, values of b are plotted along the abscissa, and 


values of the sum of the variances are plotted along the ordinate. 
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The grid superimposed on each graph is a one-inch square grid. The 
scale of the abscissa, x scale, the scale of the ordinate, Y-SCALE, 
the initial reliability, P(0), and the total number of trials inten 
years, N, are lettered at the bottom of each graph. These values are 
written in powers of ten, The number directly next to the letter E is 
the power of ten by which the main number is to be multiplied. For 
example, the number 5. 00E-01 is read as 5.0 x 10 eee Sane 
number 2. 00E+02 is read as 2.0 x 10° or 200. The numbers just to 
the right of each plot designates the sample plan that was used to obtain 
the plot. Suppose the sum of the variances for sample plan one is 
desired when Py =- i} N=] 200, and b=. 1. The graph Jor this case 1s 
found on page 20. The value desired can be read off the curve labeled 1. 
Similarly, the sum of the variances for sample plan six with the same 
parameters can be read from the curve labeled 6. To insure that the 
reader has found the proper values, these values have been marked 
with an x. The graphs are filed in the following order: all combinations 
of N with Po =: 7, then all combinations of N with Py = .8, and then 
all values of N with Pe oe 

From reading these graphs, it can be seen that for all values of 
the parameters sample plan six and four have the smallest sum of the 
variances, The rest of the sample plans range from next smallest to 


largest value of the sum of the variances in the following order: 


7 


three, one, two, five. For ease in understanding the results, the table 
explaining the yearly test sizes for each sample plan is reproduced on 
the following page. 

These results indicate that if 200 missiles were programmed for 
Follow on Testing in the next ten years there may be some merit to 
distributing the testing as shown below. 


Suggested allocation per year of FOT testing 
if a total of 200 trials are to be conducted: 


Year one -- 60 trials; 
Years two through six -- 20 trials; 
Years seven through ten -- 10 trials. 


These figures are obtained using sample plan four. 
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2.4 Determining FOT Test Sizes Using Model Two 
Model two uses the following expression for reliability at any 
time tf: 
et 


Rit) p= Potente t> 0 


where: 


Po = initial rehability; 
B = a known constant; 
c = an unknown parameter. 


Notice that this expression assumes an exponential decay of reliability 
as time increases. The factor B is chosen so that R(t) is always non- 
négative, The graph of e” for positive x starts at a value of one and 

is monotonically increasing as x increases, see figure 2. 4.2. 

If a eect factor was not used in the expression for reliability, then 
negative values would be obtained. Arbitrarily, a scaling factor of 
B= .01 was used. This merely displaces the curve of the second term 
in the expression downward, see figure 2.4. 2. 

To investigate Model Two, a Maximum Likelihood Estimator for 
reliability was obtained. Then, the expression for the variance of this 
estimator was determined, see section 3.4. The sum of the variances 
was then computed for all combinations of Po and N, as explained in 
section 2.2. In addition, these values of the sum of the variances were 


computed for values of c ranging from .00424 to .424. Note that if 
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larger values of c were allowed, R(t) would be negative when t = 10. 
Therefore, c has been varied over its entire range of plausible values. 

The results of the computations of the sum of the variances are 
displayed on the graphs on pages 42 through 59. The abscissa of these 
graphs represents values of c. The ordinate of these graphs represents 
values of the sum of the variances. See page 17 for an explanation of 
how to read these graphs. 

By examining these graphs, it can be seen that the sample plan 
that gives the minimum sum of the variances for all combinations of 
Po and N is sample plan six. The value of the sum of the variances 
then increases for the sample plans in the following order: plan four, 
plan three, plan one, plan five, and then plan two. 

For convenience in understanding these results, the table explaining 
the yearly test sizes for each sample plan is reproduced on the following 
page. Notice that as in the first model the sum of the variances is less 
for the plans that call for heavy testing in the early years. The plans 
that call for heavy testing in the out years result in the largest values 


of the sum of the variances. 
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2.5 Determining FOT Sample Size Using Model Three 
Model Three uses the following model for reliability at any time t: 
Rait)== Paes at 
where: 
Po = the initial reliability; 
a = an unknown parameter. 
This linear model was chosen so that the more complex models one and 
two might be compared with a simple and familiar model. 

To investigate this model, a maximum likelihood estimator for 
reliability was obtained. Next, an expression for the variance of this 
estirnator was determined, see section 3. 5. Using the method outlined 
in section 2.2, the sum of the variances was then computed as a function 
of a for all combinations of Po and N. The values of the sum of the 
variances were then determined for a range of values of a between 0 
and Py / 10. At this point, R(10) becomes nepative: thereiorne,. the 
sum of the variances has beén computed for the entire range of plausible 
values of a. 

The results of the computations of the sum of the variances are 
displayed on the graphs on pages 63 through 80. The abscissa of the 
graph represents values of a and the ordinate represents values of the 
sum of the variances, See page 17 for an explanation of how to read 


these graphs. 
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By examining these graphs, it can be seen that the sample plan 
that gives the minimum sum of the variances for all combinations of 
Po? N, and values of a is sample plan six. The value of the sum of the 
variances increases for each sample plan in the following order: 
plan four, plan three, plan one, plan five, and then plan two. 

For convenience, a table explaining the size of each yearly test 
under the different sample plans is presented on the following page. 
Notice that as in the first two models the minimum sum of the variances 
oer for the plans that test heavily in the early years and moderately 
in the out years. The plans that call for heavy testing in the out years 


have the largest values of the sum of the variances, 
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2.6 Comparison of Models One, Two, and Three 

The sum of the variances for all three models was minimum for 
sample plan six. Sample plan six calls for extremely heavy testing in 
the early years and very little testing in the out years. Sample plan 
four results in values of the sum of the variances that are next 
smallest to the values for plan six. However, plan four is very 
similar to plan six. Note that plan three also calls for heavy testing 
in the early years and moderate testing in the out years. Sample plan 
three is different from plans four and six in that the testing in the 
early years, although heavy, is more moderate than that called for 
in the early years by plan six or four. The testing required in the out 
years by plans four and three is moderate. However, the scheduled 
testing is not as moderate as for sample plan six. For all cases, i.e., 
all models and values of the unknown parameters, the sample plans that 
call for moderate testing in the early years and heavy testing in the out 
years (plans two and five) result in values of the sum of the variances 
that are a maximum. 

Testing heavily in the early years rather than testing uniformly 
throughout the years or testing heavily in the out years has some 
appeal to the practical. Since missile systems are Se eerarnee for 
an operational life of about ten years, planners might be reluctant to 


launch any large correction program determined necessary by testing 
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in the out years. Rather than spending money ona retiring system 
the planners might prefer to spend the money speeding up the phase 

in of the successor system. However, efforts along this line are 
usually not successful and in all likelihood an unsuspected discrepancy 
in a missile system that shows up in the out years will not be 
corrected, 

Heavy testing conducted in the early years will allow for much 
higher confidence in the predictions of reliability for the out years, 
From these predictions, a potential discrepancy in the system that 
might not be noticed until the out years under other sample plans may 
be uncovered early in the system's life. Because of this, corrective 
action can be taken while time is available and planners are more 


willing to invest in the system. 
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3. ANALYSIS AND DISCUSSION 


3.1 Mathematical Support of the Inventory Model 

In the final analysis, the number of missiles from a system counted 
on by the strategic planner will be determined by the number of missiles 
available and the 90 percent lower confidence limit for system 
reliability. In other words, if the 90 percent lower confidence limit 
of reliability is .7 and there are 200 missiles available, the strategic 
planner will count on .7(200) = 140 missiles. 

When conducting an OT or FOT of a missile system, a missile is 
fired under the most realistic conditions attainable in peacetime and the 
impact of the missile on the test range is monitored to determine if 
the missile firing obtained the desired results. Whenever a missile is 
tested, the lower confidence limit of reliability comes a little closer 
to the observed success rate. : In fact, if all the available missiles 
were tested, then the lower confidence limit for reliability would be the 
observed success rate. A plot of the 90 percent lower confidence Jinut 
versus the number tested for observed success rates of .7, .8, andi.9 
is shown in Figure 3.1.1. 

The crux of determining the number of missiles to test is depicted 


here. Obviously, we want the lower confidence limit of reliability to 


The observed success rate is merely the number of successful 
firings divided by the total number of firings. 
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be close to the real reliability of the system. However, to make the 
lower confidence limit for system reliability the same as true system 
reliability, we must test all the missiles, Certainly this is not the 
solution. The problem of where to stop testing could be solved by 


observing the graphs in Figure 3.1.1 and noting that one is beyond 





i : : 
the "knee" of the curve if testing is conducted until the lower confi- 
dence limit is within about .1 or .09 of the observed success rate. 


Let us investigate just what effect a criterion such as this will 





have ona fixed inventory of missiles. In this discussion, the following 


common notation will be used: 


| M = the total number of missiles in inventory; 
n = the size of the test; 
lL, = the 90 percent lower confidence limit for reliability. 


This number is determined by the size of the test 

and the observed success rate, 
Assume that M is 200 and that the observed success rate of the missiles 
tested is running in the vicinity of .8. To satisfy the criterion that the 
lower confidence limit for reliability be within .09 of the observed 
success rate, i.e., .71, will require approximately 50 tests and the 


strategic planner will use (200-50) .71 = 106 as the expected number of 


ithe "knee'' of the curve is a term used commonly by some 


economists and is loosely defined as the point on the curve where the 
marginal increase in utility per resource expenditure is relatively low. 
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20 
VALUE OF L(M 


10 





reliable missiles. However, if, under the same circumstances (i.e., 
observed success rate of . 8) the system was tested only 30 times, the 
lower confidence limit for reliability would be .675 and the strategic 
planner could count on . 675(200-30) = 114 as the expected number of 
reliable missiles, Not only is this a gain in the expected number of 
reliable missiles in inventory, but it is also a monetary gain af the cost 
of testing 20 missiles. Perhaps one more such excursion will illumi- 
nate more fully the costs of the criterion outlined above. Assume that 
M is 150 and the observed success rate is running about .7. To satisfy 
the criterion that testing be continued until the 90 percent lower confi- 
dence limit on reliability is within .1 of the observed success rate, 
i.e., a lower confidence limit of .6, will require 50 tests and the 
strategic planner will count on .6(150-50) = 60 as the expected number 
of reliable missiles. Under the same circumstances, if only 30 
missiles had been tested, the lower confidence limit for reliability 
would be .565 and the strategic planner would count on . 565(150-30) 
= 68 missiles. Again this is a savings of eight more missiles in the 
expected number of reliable missiles in inventory and the cost of 
testing 20 additional missiles. 

For the reasons noted in the last paragraph, an alternate criterion 
for determining the number of missiles to test might be to choose n to 


maximize the quantity Q in the following expression. 


Q= L(M - n) 
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Notice that when n is zero, Lis zero, and therefore Q is zero. 
Whenever n is between zero and M, Qis positive. Since Q is zero when 
n equals M, we know that Q has a maximum value for some values of 
nin the interval 0 -M. In an effort to show that many complex 
problems do not require extensive and high-powered mathematics, the 
value of n that maximizes Q has been solved graphically. 

By using semi-log paper and using values of L(M-n) as ordinates 
versus values of n along the abscissa, a plot of the values of Q for 
various success rates and values of M can be obtained. This graph 
is shown in Figure 3.1.2. Plots of the value of Q for all combinations 
of M =50,- 100, 200, and’400 and success rates of . 59.6, .7, 73, 29 
are displayed, Notice that for values of M greater than 100 the value 
of n that universally maximizes Q is from 33 to 38 missiles. Thus, 
even though the success rates will be unknown prior to experimentation, 
it so happens that the optimum sample size n is basically a function only 
of Mand the contidence level. (This can be séen from Figure 3, )o2 5) 
Consequently, the following procedure can be implemented. This 
procedure is that for M between 200 and 400 and success rates greater 
than .5, the test size should be 33, For information, a table demon- 
strating the small increase in the 90 percent lower confidence limit 
for variations in test size about this "optimal'' number is shown on the 


following page. 
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3.2 Description of Missiles Deployed Model 

One problem with the model presented in the first section is that 
we can not expect to use all of the missiles in inventory. More than 
likely the Polaris system will use only the missiles that are already 
deployed at the outbreak of the exchange. 

An appealing model that could be used to determine the optimal 
test size, considering only use from deployed missiles, can be con- 
structed if it is assumed that our defense resources have been allocated 
in a near optimal manner. In other words, money was spent on the 
Polaris program until the cost of increasing our defense posture 
through Polaris was more than the cost of increasing our defense 
posture by an equal amount using another system. Realize now that 
testing a system merely allows us to count on a larger number of re- 
liable missiles, ; The expected number of reliable missiles can also 
be increased by procuring a larger number of systems. The cost of 
increasing the expected number of reliable missiles through procure- 
ment is called the marginal cost of reliable missiles through procure- 
ment. The marginal cost of increasing the expected number of reliable 
missiles through testing can be computed using the following formula. 

Of course, if, on subsequent testing, the observed success rate 
was lower, the expected number of reliable missiles used by the 


planner could conceivably be lower in this case. In this exposition, we 
are assuming that success rate is nearly constant. 
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MC = AL(N)/C 


where: 

MC = The marginal cost of increasing the expected 
number of reliable missiles through testing. 

SL = The increase in the 90 percent lower confidence 
limit for reliability caused by the last test of the 
system. 

N = The average number of missiles deployed at any 
time; 

LG = The cost of testing the missile. 


Now testing should be conducted until the marginal cost of increasing 
the expected number of reliable missiles through testing is equal to 

the marginal cost of increasing the expected number of reliable missiles 
through procurement, 

An immediate argument against this model is that to increase the 
number of reliable missiles through procurement would require a new 
submarine. This is an extremely costly process and would therefore 
lead to excessive missile testing if the above criterion was established. 
However, the dilemma can be resolved by considering the marginal 
cost of increasing the yield over the target by any system that com- 
petes with Polaris. The lowest cost in this group should be used in 
place of the marginal cost for Polaris. Thus, one could Say that 
testing will stop whenever the cost of increasing the expected yield of 


Py ierd is introduced here merely to have a common ratio between 
Polaris and the systems competing with Polaris. 
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the Polaris system from testing equals the cost of increasing the ex- 
pected yield of any competing system through procurement. In this 
case, MC would be computed as follows: 

MC = AL(N)(Y)/C 
where Y is the yield from a Polaris: missile, 

An explicit solution for n is not included here because the marginal 
cost of procurement for Polaris and the systems competing with 
Polaris is not known to the author. However, these figures have been 
computed; and once they are used, an explicit value for optimal n 


follows immediately. 


22 





3.3 Mathematical Support of Model One 

Consider a collection of N missiles that are characterized by 
lifetimes x, : xX, ae x where x. represents the time from ‘initial 
inspection and release until the ‘a item deteriorates to an unacceptable 
state. If it is assumed that the initial states of the missiles are the 
same, then the X.'s are non-negative independent, identically distributed 
random variables. Suppose that there are ten distinct times that ob- 
servations are to be taken from the collection of missiles. Also, 
assume that K. items are to be observed at time t.. These observa- 
tions can be summarized by the random variables yy , Y, ; Y, pene K 
where; 

Y = { 1 if the missile system fails 
: 0 if the missile system is successful 

On the basis of this information, it is desired to estimate R(t), P 

(a missile's lifetime is greater thant). The estimation procedure 


which is proposed is of the following form. For each timet, the 


estimate of R(t) is given by 





nh. 
t 1 
Il N-n - Y 
i=l Bat K=(n, +1) e 

R(t) = 
N(N -n,)(N-n,)...(N-n,_)) 
where: 
= 
76 
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where: 


2 
Nii = ek i= eee 
r=] 7 


K 


16 


no, of trials in year r 


From this, it is obtained at once that: 


t 


E(R(t)] => 


(N-n,)(N-n,)... (N-n,_)) 


t 2 

2 a EA ie ce Ne) es is Be 
ECR (t= =< 
[N(N-n,)(N-n,)...- (None 


2 2 
+K.p,(i-p)+K, (l-p.) J 


z 
[N(N-n,)(N-n,)... (N-n, 4] 


and the variance of the estimator is given by 


Var R(t) = ECR(t)“] 5 [ECR(ty]] 


Note that Model Ome can be defined as: 


th 
R(t) = R(t-1) | ie wo 
N—no. of missiles tested prior to time t 


By considering numerous different possible results of testing, it was 
observed that this estimator tends to be more conservative than the 
Maximum Likelihood Estimator of reliability in the early years, In 
the out years, this estimator tends to be pessimistic. In addition to 
the above properties, this estimate lends continuity to the estimates of 


reliability. The excursions listed on the next page illustrate these points, 


es 





Excursion 1]: 


N= 1007 K. =- 10, for alli. There are two failures in each test. 





Year Model One Estimate MLE* 
1 R(t) = ee = .98 R(t,) = aeons = .8 
2 te 98 (1-aa) = .956 Rit = <8 
3 R(t.) = .956(1-aa)=.932 R(t) = .8 
4 R(t,) = 932 (1-2) =.904 R(t) =: 

5 R(t.) = .904(1- 25) =. 874 Rio = 38 

; R(t.) = 874(1 ~ ae i 837 R(t.) 28 

7 R(t.) = ,837(1-ga) =. 795 R(t.) = .8 

8 Re) = (795 ee, 7 Ries = <8 
8 30 8 

9 Rity) = TAL (Lae =. 667 R(ty) = .8 

10 R(t) 9) = 667 (1 - a) =, 532 R(t, 0) = 8 


* Estimate of Reliability using only the yearly test information. 
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Excursion 2: 
We 1004 K. = 10> forall 1; 
i 
Year 1 Zz 3 4 5 6 4 8 9 10 


LY. 2 ] z 1 4 Z 0 3 l 5 
i 


Note that the average number of failures per year is two. 


Year Model One Estimate MLE* 

~ TOO te 3 S 

l R(t)) aoe |S Pie ar R(t,) = ae 
~ l A 

Z R(t.) =se0?) tl - 90) =e Oo R (t,) = a9 
~ Zz. A 

b R(t.) Sere ee nel = a) = ,954 R(t.) = wc 
~ l A 

4 R(t,) = 4954 (1 - 7) =, 94 R (t,) = 9 

5 R(t.) = .94 (1 pea 88 ts ee 

a rn hae 3. (t,) =. 

a Rie) = eScuer eees me Aces Rij = 6 

Choa 7 50) 7 ( 6) a 

7 R(t.) =) 355 R(t.) =e ek) 
~ 3 A 

8 R(t.) = .835(1-se) = .752 Ric) eae 
~ l a 

9 R(t)) = .752 (1-55) = .713 R(tg) = .9 
~ 3 a 

10 t =o. 2 = = 
Bn) i) .5 R(t) oT 


x 


Estimate of Reliability using only the yearly test information. 
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Excursion 3: 











N = 200 
Year l Zz 3 4 5 6 7 8 9 10 
K 60 20 20 20 20 20 10 10 10 10 
| DY, 9 3 4 3 2 5 0 3 4 4 
Year Model One Estimate MLE* 
~ 200.- 9 
ns = = - 5 
l R(t)) ny £955 R(t,) 8 
2 Fis) = Obl = 633 Bier ee ee 
a ao ne’ 
3 R(t,) = .933(1 eee 897 Rit.) = 8 
San ion 3 
~ 3 a 
4 (en R(t,) = 85 
5 Rib y= 87 (lee) = 849 Rise.) 2.9 
oe so eee 
~ 5 a 
= -s__ ey Ae Zz t = c75 
6 R(t,) -847:(1 oe) 725 R (te) 
i R(t.) = .725 R(t.) = 1.0 
8 R(t.) = 725 11-28) 2 OER Rit = 27 
8 ; 30 8 
Bie) = (e5edieoy = ene Rites = 26 
9 (ty) an © 20 = ° 9 ° 
R = eet ~ =), = 239 oy Ce: 
20 io Peat age = i = * 
* Estimate of Reliability using only the yearly test information. 
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3.4 Mathematical Support of Model Two 

The exponential decay model has been used on numerous occasions 
to describe reliability decay of electronic systems. To this end, 
Model Two introduces an exponential decay. However, to avoid the 


rapid decrease of reliability in the first years that ensues using the 


exponential decay form R(t) = Poe - an the following expression for 
reliability is introduced: 
R(t) = py - peo 
where: 
oa the system's initial reliability; 
B = a Known constant; 
c = an unknown parameter; 
t = Cine, te, la 2) ae Oe 


Figure 3,.4,1 illustrates that aor greater than or equal to one for 
all positive values of c and t. Therefore, if a value of B that is greater 
than or equal to one is chosen, negative values of reliability will result. 
To avoid this, Bhas been arbitrarily set equal to ,0l. This value 
merely displaces downward the value of the second term in this ex- 
pression, With this value of B, the expression for reliability will be 
positive for all combinations of t from zero to ten and c from . 00424 to 


404, 


Because of the invariance principle, we may obtain an estimator 


for c, call it ¢; and then the estimate for reliability can be obtained 
by substituting ¢ for c. We will, therefore, assume that Py is known 
and obtain a Maximum Likelihood Estimator for c. As a reminder, 
all testing done in year iis designated as testi. Each individual firing 
ee ; ; : .th 

of a missile is called a trial. K, is the number of trials inthe i test, 
and s. is the number of successful trials in test 1. 

The MLE is obtained in the following manner. The probability that 

th 

S successes are observed during thei test can be expressed by: 


K: Ke - S 
i S i 
P'{no. of successes = 5] = (4) (R(t)) (1 = Rit) 


K. -s 
i 


I 


Ky Cl.s ct 
(See ee = aaa 
0 0 

Let L(c) = P[no. of successes = 5], Then 


K: t 
In L(c) = In(s") + s In (p, - Be) + (K, = 5) In (1 - Py + Bey 


t 
yin bie) JepBe” (K, - 5) (tBeo*) 
ac Ct ct 
Po = Be 1 -Pyt Be 


To obtain a Maximum Likelihood Estimator ofc, set this equation equal 

to zero. The solution to this equation is ¢. Doing this yields 

Ke - Ss. 
Po i 


1 


expanding in a Taylor series 
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a> 
ul 
| 
_ 
=) 
t 
. @ 
_ 
= 


AD ]-t wo? -[-n(- 11} 
































2 ‘ 5 2 a8 
O ] . P. 
ape "5 = 7 : 5 - — 1 } 
2 
0 Po 3P 9 
a 2 ine Sy 
Z B ie 2 5 } 
t 2 3 
° Po Po 
<2 me ee 5 . 6 
ee gs ee Se os re a 
z | 2 3 ie 3 5 9 6 } 
Po Po Po Po Pg 
r 
r =a r 1 
Now since p= aes then E P. |= = 
Ee K, 


random variable with parameters P. and K.. The method of obtaining 
the expected values of the powers of s using factorial moments is 


illustrated in Appendix 1. 


Using these values, the mean and variance of the estimator for c 


may be approximated from the following expressions. 


l ror ease in writing these long equations, P. has been used for 
Rte. 
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p 
: 1 Ont, 24 1 1 2 
Bie | E[s.) + —>—> =I, | 
eran Bet Beene : 2K.“ p ; 
i 0 
l 3 
Si ere. Jie ere } 
i Po 
Pp 
1 pen 1 1 2 
a eee ee oa bee Ot gee, ES Pe 
i i 0 ZK. p 
r ~~ 0 
: Hedin, = aie: he 
3 eo rage EE es 
i Po 
2 
+ 3(K. (K, - 1)p,” +K.p,) -2(Kp,)] ee, 
p p 
alee, Sal 2 Po 0 l 1 Z 
Eé"]2—> [wo ge - 2 ye [gee Els] + —p az Els] 
ty x 0 2K. p 
1 i -“O 
2 
a : E[s 1 | aE Pie) 
aE ES ae ; 
Po Po 
l 
+ E[s. ] + Es E[s.°] 
ag * iene 
Po i Po 
1 1 6 
om: g Fle, 1 +g ~ E[s, ]+ } 
i Po i Po 
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1 pa z 
aes eS ae es 





Le B B K. (Kp) 
‘ pa 
1 
ee Mie Tee ces 
2 2 UA A ee nee 
i Po 
ee ie ik Nome Oe (Sete 
Maes ae ae eee CK (K;-1)p, ee 
i Po 
= 2K. Pod tess J 
ee aici — inte ie 
Siar are, te 
i Po 
pe eerie Se See 
7 oe US pore, 
I; Po 


2 
3 B : 
[K,(K,-1)p + K, p,] -2Kp,] 


1] 4 
+ mera ae’ {K. (K. - E) (K. - 2) (K. - 3)P. 
12K. Po 


3 zy 
6[K. (K, - 1) (K, - 2) P, + 3[K. (K. - 1) p; 2, + K. p, ] 
2K 11 [ K.(K 1) e 


6K p, } + — 


Var (2.7) 2 fe 


103 


3.5 Mathematical Support of Model Three 
Models One and Two involve fairly complicated expressions for 


R(t). It might be illuminating to compare these more complicated 


models with a simple linear model. Therefore, the following expression 


for reliability is proposed. 
R(t) = Po - at 
where: 
Pas the initial reliability of the system; 
a = an unknown parameter; 
t S41me, 1:6 ee 244 0e 
A Maximum Likelihood Estimator for a will, by the invariance 
principle, result in a Maximum Likelihood Estimator for reliability. 
The MLE of a is found in the following manner. The probability that 


S successes are observed in K. trials can be expressed as: 
i 


K. -s 
RG le Ree 


P[s. =s] 


K P Ke = Ss 
= (81) (py - at)” (1 - py + at) 


Calling this expression L(a) and taking logs: 


ictedettay othe ety Genes (py - at) + (K, - s) In(1 - py + at) 


and 
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Setting this expression equal to zero and solving for a gives 


— Sop o 7 Po Ss 
a Kat on re oS 
1 11 
and 


BEfa) =a Varifa) = —; (py - at) (1 - pp + at) 
Kt 


follows directly from the expression for a. 


APPEND 


FACTORIAL MOMENTS OF A BINOMIAL RANDOM VARIABLE 


The moments of a binomial random variable, s, may be generated 


using the probability generating function 
K 
M , (2) = (q + pz) 
where K indicates the number of trials and 


q 


Il 

— 

' 
4°) 


For simplicity in notation, let 














G = M,(z) 
and 
pe toe 
Now 
3G K - 
Se ete ee P/,- , — *P 
Sle Kea2) Z Z 
5 = K(K-1)(q+ pz) “p = K(K-1)p 
oz Zonal z= 1 








: x : 
Since the term (q+ pz) will always reduce to one when z is set 


equal to one, we see a recursion relation, namely: 


SAK Sl) eK Sn 51) 5 


OZ Z= 
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We may now deduce that: 


Papers). (esl) ] = w(K = t)sc. (= (a Pp 


Proceeding from here, the expected values are determined below. 
2 
Efs(s -1)(s - 2)] = K(K - 1) (K - 2) p 
me = 3 
Ss -38 +28 = K(K - 1)(K - 2)p 


3 2 
, 3s =KYK =I) (KR = 2)p +36 -26 


E(s (s - 1)(s - 2)(s - 3)] = K(K - 1)(K - 2)(K - 3) “i 


a eee eis oe se eee ee a 


Sik hee ie ee Se ee 


E{s(s - 1)(s - 2)(s - 3)(s - 4)] = K(K-1)(K-2)(K - 3)(K-4p 


oo ios ase se ee ee i ey tn eee 


4 2 
3 hie uw 1ytk 6 2k ee 10s eager (60s 20246 


Bl sis ly(s—2)ie- S\(s= 4)) = Rik = 1K - 2k < ok = ak sae 


3 
3° - ie + aa” - 2258 + 2743° - 120s 


= K(K - 1)(K - 2)(K - 3)(K - 4)(K - ta 


3° = K(K - 1) (K - 2) (K - 3) (K - 4) (K - 5) p” 


tr ise” - Scie 2253" - 745" + 1208 
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APPENDIX II 
COMPUTER PROGRAMS 





SRE RE RR AINNNNNNNNNO SO 
N ™ 


— p= 


420, 10X,E9.8///) 


(11) ,EPP(11),EPS(11) 


9 
0 
1 
-SUM-AK(JP,J0O) #(1--P(JP,JM)))/(AN-SUM) 
S 
( 
( 
9 
A 


Qn 

Wo Fe 

ee D000 

—O ore WWW) 
Mr = Im — ON NaNO aw xmor ~De za 


JO)#(1--P(1,IM)))/AN) 
) 
) 
A 
9 


L*JN) 


[eet ~O "ox 
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